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ABSTRACT 

In  this  paper,  we  discuss  the  Stochastic  Resonance  (SR) 
effect  for  the  binary-input  ternary-output  (2,3)  Discrete 
Memoryless  Channels  (DMC).  The  (2,3)  DMC  is  of  great 
interest  for  various  digital  communications  channels, 
including  acoustic  underwater  communication.  We  focus 
on  the  capacity  maximization,  brought  about  by  additive 
Gaussian  noise,  for  the  (2,3)  DMCs.  We  analyze  the 
capacity  of  the  threshold  binary  communication  channels 
in  the  presence  of  Additive  Gaussian  Noise  (AGN).  In  the 
particular  case  of  binary  asymmetric  channels  with 
symmetric  erasures  (BAC/SE)  we  derive  the  optimum 
noise  level  in  order  to  obtain  the  maximum  capacity.  The 
stochastic  resonance  for  the  general  binary  erasure 
channel  is  compared  with  those  of  binary  symmetric 
channels.  For  the  general  case  (2,  n)  DMC,  the  Pinsker 
and  Helgert  capacity  bounds  were  used  for  the  study  of 
the  stochastic  resonance. 

KEY  WORDS 

Mutual  information,  stochastic  resonance,  binary  channels 
with  erasures,  threshold  communication  systems 

1.  Introduction 

In  this  paper,  we  discuss  the  Stochastic  Resonance  (SR) 
effect  for  the  binary-input  ternary-output  (2,3)  Discrete 
Memoryless  Channels.  The  (2,3)  DMC  is  of  great  interest 
for  various  digital  communications  channels,  including 
acoustic  underwater  communication  [12].  We  study  the 
SR  impact  upon  the  performance  limits  of  distributed 
underwater  wireless  sensor  networks  operating  with 
limited  transmitted  power  and  computational 
environment.  In  general,  the  SR  phenomenon  is  a  non¬ 
linear  effect  wherein  a  communication  system  can 
transmit  the  information  with  improved  efficiency  in  the 
presence  of  the  additive  Gaussian  noise  (AGN)  [1]  and 
[11].  We  focus  on  threshold  communication  systems  due 
to  the  underwater  environment,  noncoherent 
communication  techniques  are  affected  both  by  noise  and 
threshold  level  [5]. 


Various  performance  metrics  in  the  presence  of  SR 
such  as  signal  to  noise  ratio,  mutual  information,  and 
channel  capacity  improvement  under  a  certain  range  of 
power  noise  levels,  were  discussed  in  [2].  In  [9],  [13]  and 
[14],  the  SR  effect  is  observed  on  the  binary-input  binary- 
output  (2,2)  DMC  capacity,  where  maximum  capacity 
occurs  at  an  optimal  power  (variance)  value  of  the  AGN 
in  relation  to  a  given  threshold  level. 

Following  Chapeau-Blondeau  [2],  we  analyze  the 
physical  communication  model  of  the  (2,3)  DMC  for 
threshold  based  SR  due  to  additive  Gaussian  noise.  Our 
major  contribution  consists  in  deriving  the  optimal  noise 
level  required  to  obtain  the  maximum  capacity  for  a  given 
threshold  decision  level.  Based  on  the  work  derived  in 
this  paper  one  can  find  the  maximum  capacity  of  the 
BAC/SE  in  the  context  of  SR.  We  use  the  Helgert  and 
Pinsker  capacity  bound  approximations  [3], [4], [8]  to 
study  the  capacity  behaviour  for  a  distributed  cooperative 
wireless  sensor  underwater  network. 

The  paper  is  organized  as  follows.  In  Section  II  we 
provide  the  background  for  the  mutual  information  and 
capacity  of  the  (2,  3)  DMC.  The  effects  of  the  SR  and  the 
physical  model  of  the  (2,  3)  DMC  communication  channel 
are  analyzed  in  Section  III.  In  Section  IV  we  derive  the 
optimal  noise  level  and  the  optimal  threshold  level  in 
order  to  obtain  the  maximum  capacity.  The  conclusions 
and  future  work  are  given  in  Section  V 

2.  The  (2,3)  DMC  Mutual  Information  and 
Capacity 

The  (2,3)  DMC  (M:  X  ->  Y}  is  characterized  by  a  binary 
input  random  variable  X  —  {1,—1]  ,  a  ternary  output 
random  variable  Y  =  (1,0,  — 1}  and  a  (2x3)  channel 
transition  probability  matrix  M  defined  by  M  — 

„  ).  The  entries  of  this  matrix  are  the  p;i 

P 12  P22  P322  1 

which  represent  the  conditional  probability  between  each 

output  and  input  values  as  represented  in  Fig.  1. 
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Figure  1.  (2,  3)  DMC 

Associated  with  Fig.  1  we  have  the  following  conditional 
probabilities  between  binary  input  and  ternary  output: 

Pn  =  P(Y  =  1\X  =  1),  p21  =  P(Y  =  0\X  =  1), 

Psi  =  P(Y  =  -1\X  =  1),  p12  =  P(Y  =  1\X  =  -1), 

P22  =  P(X  =  0\X  =  -1),  p32  =  P(Y  =  -1\X  =  -1). 

Since  M  is  a  stochastic  matrix  we  have,  p31  =  1  —  Pll  — 
p2i  and  p32  =  1  -  P12  —  P22  • 

Defining  P(A  —  i)  —  p(X;),  i  —  1,2,  and  P(Y  =  j)  = 
p(y;)  ,  j  —  1,2,3,  then  the  mutual  information  between 
the  binary  input  X  and  the  ternary  output  Y  is 

I(X,Y)  =  H(Y)  -  H(Y\X)  (1) 

where  H(Y )  the  entropy  function  of  a  Y  probability 
distribution  defined  as 

h(y)  =  -  Zj  viyj)  log2  p  (y/)  (2) 

and  H(Y /A),  the  conditional  entropy  defined  by  the 
following  relation 

H{Y/X)=Zi,iv{xl,yj)\o  g2^  (3) 


I  (X,  Y)  = 

Ei=1Tj=1P(xi)  p(yj  l^i)l°g2  rj££ljltk)  (5) 

In  order  to  calculate  the  capacity  of  the  (2,  3)  DMC 
channel,  we  want  to  maximize  the  mutual  information 
I(X,Y )  [6],  [10].  Based  on  (5),  we  see  that  I(X,Y )  is  a 
function  of  pfe)  and  p(x2)  since  the  p(y/|*;)  =  P/j  are 
fixed:  I(X,Y )  =  f(p(x1),p(x2)).  Therefore,  in  order  to 
calculate  the  capacity  of  this  channel,  we  need  to 
maximize  I(X,Y )  over  all  distributions  of  the 
form  (p(*i), p(x2)).  Since  p(Xj)  +  p(x2)  =  1,  we  set 
x  =  pOO,  p(x2)  =  1  -  x,  giving 

C(Xr)  =  max*  I(X,Y)  (6) 

and  finally  replacing  (5)  in  (6)  we  have 

C(X,  Y)  = 

max,  ZU 1U  Pi*d  PiVj  \xt)log2  (?) 

While  the  general  capacity  solution  for  (6)  is  not 
known,  in  [7],  a  closed  form  solution  is  given  for  the 
capacity  of  the  binary  asymmetric  erasure  channel  with 
symmetric  erasure  (BAC/SE).  The  BAC/SE  is  a  particular 
case  of  the  (2,  3)  DMC  with  the  following  constraints: 
P21  =  p22  and  Pll  >  p12  (without  loss  of  generality)  as 
mentioned  in  [7]. 

In  [7],  using  the  Kuhn-Tucker  condition,  the 
following  solution  is  found  for  the  BAC/SE  channel 
capacity: 

C(X,  Y)  = 


Using  (2)  and  (3),  we  can  rewrite  the  mutual  information 
as  follows 


1{X,  Y)  =  ZijPtXi.yj)  log2 


ptxi)p{yj) 


(4) 


where 


where 


_ feP32~Pl2 _ 

Pll-Pl2~k(.P31~P32) 


(9) 


Since  p(.Xi).p(yj\Xi)  =  p(xuyj)  and  p(y;-)  = 
1,1=1  PiXj \xk)  P(.xk)  we  finally  have 


k  =  { 


PllPllP31P31\Pll-Pl2 

Pl2Pl2P32P32' 


(10) 
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Using  the  closed  form  solution  (8),  we  will  examine  the 
stochastic  resonance  that  arise  in  the  (2,3)  BAC/SE 
channels. 

3.  Stochastic  Resonance 

Following  the  work  of  [13]  we  will  examine  what  is  the 
effect  of  AGN  on  the  capacity  of  the  (2,3)  BAC/SE  (as 
shown  above)  derived  in  [7].  The  input  to  the  threshold 
communication  channel  is  the  signal  x  that  takes  the 
binary  values  -1  and  1  with  the  probabilities  p(x F)  and 
p(x2).  The  physical  model  is  represented  in  Fig. 2  where 
in  the  presence  of  the  AGN  n,  the  received  signal  y  is 
given  by 

y  —  x  +  n  (11) 

The  physical  communication  channel  presented  in  Fig.2 
contains  a  threshold  decision  block  function  t(0),  where 
9  represents  the  threshold  value.  Based  on  the  threshold 
level  9  and  the  noise  level,  the  received  signal  y  is 
converted  into  a  discrete  random  variable  y  taking  on  the 
values  of  1,  0,  or  -1. 


n  ~N(0,  er2) 


Figure  2:  Physical  model  ot  binary-input  ternary-output 
channel  with  zero  mean  Gaussian  noise 

For  the  ternary  output,  the  threshold  function  is  defined  as 
follows: 

f—1  x  +  n  <  —  9 

t(6»)  =  0  — 9<x  +  n<9  (12) 

(  1  x  +  n  >  9 

By  using  the  threshold  level  we  will  determine  the 
analytic  expression  for  each  of  the  conditional 
probabilities  p11p21,p31p12p22,p32  between  the  binary 
input  and  the  ternary  output.  We  assumed  the  noise  n  to 
have  a  Gaussian  distribution  with  mean  p  =  0  and 
variance  er  >  0  with  the  following  cumulative  density 
distribution 


Pn  =  P(1\T)=P(Y  —  1\X  =  1)  = 

Pr(x  +  n  >  9  \x  =  1)  =  1  —  Pr  (n  <  —  1  +  9)  — 
1-Fn(-1  +  9)  (14) 

Using  a  similar  approach  we  have 

fp21  =P(0\1)=Fn(-1  +  9)-Fn(-1-9) 
p31=P(-l\l)  =  FN(-l-9) 

<  P12  =P(ll-l)  =  l-F„(l  +  9)  (15) 

p22  =  P(0|  — 1)  =  Fn(1  +  9)-  Fn(  1  -  9) 
lp32  =  P(-l\  -  1)  =  FN(1  -  9) 

Since  F{—x)  —  1  —  F(x)  ,  and,  according  with  (14)  and 
(15)  the  following  relation  stands 

Pn  =  P32 )  P21  =  P22 )  P31  =  P12  (16) 

When  noise  is  modeled  as  a  standard  Gaussian 
distribution,  and  since  1  —  9  >  —  1  —  9  ,  we  obtain  that 
Fn{  1  -  9)  >  Fw(— 1  -  9 )  and  Fw(  1  +  9)  -  FN{  1  -  9)  = 
Fn{9  —  1)  —  F);(—  1  —  9),  respectively.  These  are  exactly 
the  values  which  correspond  respectively  to  pn  >  p12 
and  p21  =  p22  which  characterize  the  capacity  of  the 
BAC/SE.  At  this  point,  we  make  the  fundamental 
observation  that  the  (2,3)  physical  communication 
channel  presented  in  Fig.2  is  a  BAC/SE  channel  in  the 
presence  of  AGN.  This  motivates  our  work  to  study  the 
capacity  SR  behavior  of  this  channel  which  was  not 
presented  in  [11].  The  effect  of  other  types  of  noise  on 
this  communication  channel  is  left  for  future  work. 

Using  (13),  (14)  and  (15),  we  can  re-write  the  (2,  3) 
DMC  matrix  as: 

=  (Pn  P21  Psi\  (1  -  Fn (-1  +  9 ) 

VP12  P22  P32)  V  1  ~  Fw(l  +  9) 

FN(-1  +  9)  -  FN(-1  -  9)  Fn(—1  —  0)\ 

Fn(1  +  9)-Fn(1-9)  Fn(1-9)J[  ' 

By  using  (16)  we  have  k  —  1  in  (9)  and  x  —  1/2  in  (10). 

The  capacity  of  the  BAC/SE  channel  could  then  be 
rewritten  below  in  (18)  and  it  is  plotted  in  Fig. 3  and  Fig. 4: 

C(X,  Y)  = 

Fn(1  -  S)log2  (0  5(pN(1_e)_FN(_1_e))+FN(_1_e))  + 

Fn(-1  -  Uk-^WJ  (l8) 


Fn(0)  =  Fr(n  <  9)  =  e~2a2  dt  (13) 

Therefore  we  have 


We  observe  the  phenomenon  of  SR  on  capacity  over 
a  specific  range  of  9  and  noise  power  a  as  it  is  plotted  in 
Fig. 3. 
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been  proven  that  any  (2,  n)  discrete  memoryless  channel 
with  2  inputs  and  n  outputs  with  the  transition  matrix 

Pi,i  ■■■  Pn- 1,1  Pn,l'' 


M 


-( 


P  1,2 


Pn-1,2  P, 


n,l\ 

’nJ’ 


where  pnl  =  1  —  p1±  — 


- Pn— 1,1  and  pn  2  =  1  -  plj2 - p„-i,2  ,  can  be 

approximated  by  :  a  lower  bound  called  the  Pinsker 
bound,  that  we  denote  by  L,  and  a  upper  bound,  the 
Helgert  bound,  that  we  denote  U.  The  Pinsker  and 
Helgert  bounds  are  defined  as  follow: 


L  <  C  <  U  where 


L  = 


8 ln(2) 


|pt,l  Pi, 2  I  I  +  |Pn,l  Pn,2  I 


Figure  4. Capacity  of  BAC/SE  for  0=  {0.5,  0.6,  1,  1.5,2, 
2.5,  3,  3.5,  4} 

To  be  more  specific  it  is  important  to  note  that  the  SR 
appears  only  for  9  >  1  as  it  is  plotted  in  Fig. 4.  For  the 
noise  power  a  £  [5,10]  and  the  thhresold  level  9  >  3  in 
Fig.  3,  the  capacity  is  vanishing  to  zero.  Similar  results 
were  reported  before  in  [1],  [2],  and  [9]  for  the  (2,2) 
DMC.  While  these  results  are  in  concordance  with  the 
intuition  that  increasing  the  noise  power  on  the  channel 
will  deacrese  the  capacity,  the  opposite,  when  the  noise 
power  decreases  the  capacity  is  also  decreasing  will  be 
detailed  in  the  full  paper. 

4.  Stochastic  Resonance  and  Capacity 
Bounds  for  (2,n)  DMC 

Due  to  the  non-existence  of  a  closed  form  expression 
for  a  (2,n)  capacity  channel  in  general  and,  the  non-linear 
expression  of  the  (2,3)  capacity  channel  in  particular,  we 
will  use  the  bounds  (upper  and  lower)  developed  in  [4]  for 
the  particular  case  of  the  (2,3)  DMC  in  order  to  examine 
the  phenomenon  of  stochastic  resonance.  In  [4], [8]  it  has 


U  = 


n- 1 

\ 

max  y  pL1 

;  ^  Pi,: 

!  “  /  min  (p, 

'i=l 

i  =  1 

/  \i= l 

;  Pi, 2) 


For  a  (2,3)  DMC  channel  the  above  expressions  are 
reduced  to 


81n(-2)  [IPn  -  P12 1  +  IP21  -  P22 1  +  IPsi  -  P32 1]2 

>  C  > 


ma x(pn  +  p21;  p12  +  p22)  -  min(pli;  p12)  - 

min  (P21;P22)  (24) 

We  have  shown  previously  that  using  the  threshold  model 
given  in  Fig. 2  and  modeling  noise  as  a  standard  Gaussian 
distribution  we  have  the  following  equality: 

Pn  =  P32,  P21  =  P22  ,  Par  =  P12.  P21  =  P22.  Pn  ^ 
P12,  and  p32  >  p31 

Using  these  relations  and  (24)  we  have: 

(Pn  -  P12  )2  ^  C  <  plx  -  p12  (25) 

Using  (14)  and  (15)  we  have 

^  (Fn(1  -  9)  -  Fn(—1  -9)Y<C<  Fn{  1  —  9)  — 

Fn(~  1-0)  (26) 


Finally  we  have 

L=  ^y(Fw(l-e)-Fw(-l-0))2  =/(0,<O 
U  =  Fn(1  -9)-  Fjv(— 1  -  0)  =  g(0,  a) 
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theta =0.5 


theta = I 


In  Fig.5a,  Fig. 5b,  and  Fig.  6,  we  noted  that,  just  as 
mentioned  in  [7],  the  bounds  capture  the 
increasing/decreasing  behaviour  of  the  capacity  and 
exhibit  stochastic  resonance  for  the  threshold  level  8  > 
1. 


Figure  5. a  (2,  3)  DMC  and  Pinsker  capacity  Bound 


Figure  5.b  (2,  3)  DMC  and  Pinsker  capacity  Bound 


Following  similar  work  done  in  [3],  we  will  use  the 
bounds  to  find  the  optimum  noise  given  a  threshold  that 
will  maximize  the  capacity.  The  above  lower  and  upper 
bounds  are  given  by: 

L  =  “  Fiv(-!  -  S))2and 


U=  FN(l-8)-FN{-l-0). 


Sigma 


Sigma 

thcta=2 


Sigma 


Figure  6.  Capacity(C),  Pinsker  (L)  and  Helgert  (U) 
bounds  for  different  values  of  0 

5.  A  Optimal  Values  for  Threshold  and 
Noise  Level 

We  are  looking  for  the  optimal  values  of  correlated  noise 
and  threshold  level.  As  in  [3],  our  goal  is  to  find  the 
critical  points  of  the  capacity  bounds.  This  will  require  us 
to  take  the  partial  derivatives  ^-L(0,cr)  and  ^-U(0,  cr)  , 

and  find  the  solution  to  the  equations  —  L(6,  cr)  =  0  and 

do 

^-U(0,<t)  =  0  respectively. 

The  partial  derivative  of  the  bounds  with  respect  to  cr 
will  have  the  same  critical  points  because: 

a)  =  ^  (w  - e)  -  ^(-1  -  e))'-  foe  i 

-  0)  -  Fs(-1  -  0)) 

-F„(— 1-0)) 

Thus,  solving  L (8,  cr)  =  0  is  equivalent  to.  solving 

^-U(0,  cr)  =  0  which  implies  that  they  both  have  the 
same  critical  points.  Therefore  we  will  only  focus  on 
-fj-\J(8,  cr).  We  have 

(0,<O  =  ^(Fw(l  -  8)  -  Fw(— 1  -  8)  )  = 

d  1  f1-8 

— - r=  I  e  2o-2  dt  — 

“°"cr yj(2n)  J-i-e 
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-1 


-(i -°Y 


2/(2 n) 


[(1  -8)e  2(7z  —  ( —  1  —  0)e” 


-(-i-g)2 


Solving  fore,  —  U(0,er)  =  0  ,  we  will  obtain 


,  ,-1+0-  -40 

ln^=^ 


For  9  >  1  and  a  >  0,  we  finally  have: 


20 


i  1+e ■> 

n  ti+<P 


(27) 


The  above  relation  gives  us  the  optimum  power  noise 
a  as  a  function  of  0.  In  Fig. 8,  we  plotted  (in  blue) 
a  —  f{6)  and  (in  red)  /_1(0).  The  plots  depicted  in 
Fig. 7  will  provide  two  opportunities  to  improve  the 
capacity  channel;  either  by  adding  or  reducing  noise 
depending  on  the  threshold  or  by  simply  adjusting  the 
threshold  depending  on  the  noise  level.  It  is  important  to 
mention  that  from  the  plot  of  Fig.  7,  we  see  that  as  6 
increases, a  ->  8.  That  is  because  limg^  a  —  9  —  0, 
which  means  that  8  —  a  is  an  oblique  asymptote  to 
<r  =  /(0)  and  8  —  f  (cr)  as  plotted  in  Fig.7: 


behaviour,  we  used  the  lower  and  upper  capacity  bounds, 
we  also  have  illustrated  the  stochastic  resonance.  Based 
on  these  bounds,  we  have  derived  analytically  the 
optimum  noise  power  level  depending  on  the  threshold 
level  that  will  maximize  their  capacity.  This  paper  clearly 
indicates  how  one  can  improve  the  capacity  of  binary 
threshold  communication  channel  by  changing  the  noise 
power  or  the  threshold  level.  In  the  context  of  the 
underwater  communications,  where  noncoherent 
communication  techniques  are  affected  both  by  noise  and 
threshold  level  we  derive  the  optimal  noise  power  and 
threshold  for  stochastic  resonance,  with  a  scalable  factor. 
This  research  is  a  step  forward  toward  the  design  of 
efficient  sensors  that  can  operate  in  very  noisy 
environment.  In  future  work  we  will  focus  on  the 
phenomenon  of  stochastic  resonance  in  non  Gaussian 
channel. 
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Figure  7:  Optimal  noise  power  and  threshold  for 
stochastic  resonance,  with  a  scalable  factor 

5.  Conclusion 

In  this  paper  we  have  studied  the  threshold  based 
stochastic  resonance  behaviour  of  binary-input  ternary- 
output  discrete  memoryless  channel.  We  proved  that  in 
the  presence  of  the  AWGN  the  (2,  3)  DMC  become  a 
BAC/SE  channel  for  which  the  capacity  channel 
behaviour  were  analyzed  in  the  context  of  the  stochastic 
resonance  phenomenon.  For  the  (2,  n)  DMC  capacity 
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